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^ ! Abstract 

O ' Suppose Alice and Bob receive strings of unbiased independent but noisy bits from 

some random source. They wish to use their respective strings to extract a common 
^N I sequence of random bits with high probabihty but without communicating. How many 

such bits can they extract? The trivial strategy of outputting the first k bits yields an 
^ ^ ' agreement probability of (1 — e)'^ < 2~^'^^^^, where e is the amount of noise. We show 

^ , that no strategy can achieve agreement probability better than 2~ '^^~^' . 

tyj I On the other hand, we show that when k > 10 + 2(1 — e)/e, there exists a strategy 

, ^, ' which achieves an agreement probability of 0.003(A;e)~^'^ • 2~ '^^~'^' . 

^ ■ 1 Introduction 

^ I Let X and y be strings in {0, 1}" generated according to the following random process. First, 

each bit Xi of x is chosen independently at random from {0, 1}. Then each bit yi of y is 

Q ■ independently set to equal Xj with probability 1 — e and 1 — Xi with probability e (the latter 

O . possibility indicates that Xi is corrupted). Suppose that Alice and Bob now want to agree 

. . i on a common random string with probability at least, say, 1/2. One possible protocol is for 

.^ I both of them to output the first 0(l/e) bits of their respective inputs. We show that no 

/\ • protocol can do better up to the constant factor. On the other hand we show that this gain 

j^ ■ by a constant factor can be achieved for certain values of the parameters. 

This scenario relates to the problem of extracting a unique identification (ID) string from 
process variations. Several works have proposed hardware-based procedures for extracting a 
unique, uniformly random identifying string from a digital circuit of a given type [LLG''"05, 
SHOOS, YLH"^09]. It has been proposed that such strings can be used for authentication 
and secret key generation of low-power devices such as RFIDs [LLG"''05, SD07]. 
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However, such procedures are prone to noise: Different instantiations of the procedure 
may produce shghtly different answers. Can the agreement probabihty in any pair of instan- 
tiations be improved algorithmically while maintaining the uniform distribution of the ID 
string? Our work addresses this question when the noise is random and independent across 
the bits. We note that in apphcations, the noise can be handled using other methods, for 
example by incorporating noise tolerance at the receiver end. 

The case where the goal of the two parties is to extract a single bit was studied inde- 
pendently a number of times. It is known that in this case the optimal protocol is for the 
two parties to use the first bit. See [Yan07] for references and for studying the problem of 
extracting one bit from two correlated sequences with different correlation structures. 

In [MOOS, MOR"^06] a related question is studied: If m parties receive noisy versions of 
a common random string, where the noise of each party is independent, what is the strategy 
for the m parties that maximizes the probability that the parties agree on a single random 
bit of output without communicating? [MOOS] shows that for large m using the majority 
functions on all bits is superior to using a single bit and [MOR"'"06] uses hyper contractive 
inequalities to show that for large m, majority is close to being optimal. 

The optimality of the single bit protocol for two parties and extraction of one bit implies 
that if the goal of the two parties is to maximize the expected number of bits they agree on, 
given that they output k bits, they cannot do better than output the first k bits. However, 
this analysis leaves open the possibility that there exist a strategy where the two parties may 
be able to agree on all the bits with probability as large as 1 — e. 

We prove that this is not the case: The probability of agreement can be at most 2~^'^'^^~'^\ 
In the trivial strategy, where each party outputs its first k bits, the probability of agreement 
is (1 —e)''. Figure 1 shows the ratio between the number of bits allowed by our upper bound 
and the performance of the trivial strategy, for any fixed agreement probability. 

On the other hand, when the probability of agreement is sufficiently small, an improve- 
ment over the trivial strategy is possible: When A; > 10 -|- 2(1 — e)/e, there exists a protocol 
which achieves an agreement probability of 0.003(te)~^/^ ■ 2"'^^/^^"^^. 

Our protocol is asymptotically almost optimal in the following sense. Suppose we want 
to achieve a fixed but sufficiently small agreement probability p. Our upper bound shows 
that if the trivial protocol extracts k bits, then no protocol can extract more than (l/ln2)/c 
bits. Our protocol can extract (1/ In 2 — 6)k bits for any constant 5 > 0, as long as e = e{6) 
is sufficiently small. 

Gacs and Korner [GK72] and Witsenhausen [WitTS] show that it is impossible for Al- 
ice and Bob to extract Q{n) common random bits with probability 1 — o(l) for any finite 
distribution {xi,yi), unless Xi and yi share common randomness. Our work applies to a 
specific (natural) distribution {xi,yi), but yields much sharper bounds. Maurer [Mau93] and 
Ahlswede and Csiszar [AC93] consider a different model where Alice and Bob can communi- 
cate, but eavesdroppers are present and the common random string must remain secret. In 
this model, it is sometimes possible to achieve better agreement. 
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Figure 1: An upper bound on the factor beyond which the trivial 
protocol cannot be outperformed in terms of number of extracted 
bits (for any probability of agreement). 



Notation Throughout the paper, we use n to denote the length of the correlated strings 
X and y available to Alice and Bob, k for the number of bits in their output, and e for the 
noise. The inputs x = xi . . . x„ and y = yi . . .i/n, Xi, yi G {0, 1} are chosen from the following 
distribution [x, y)^: Each pair Xiyi is independent of all the other pairs and takes the values 
00, 11 with probability (1 — s)/2 each and the values 01, 10 with probability e/2 each. 



2 The upper bound 



Consider a protocol where Alice and Bob produce k uniform bits of output. Such a protocol 
can be described by a pair of functions f^g : {0,1}" — )■ {0,1}'^ indicating the outputs 
produced by Alice and Bob, respectively. 

In our problem, Alice and Bob need to agree on an input that is uniformly random. 
We will consider a relaxed scenario where the outputs of Alice and Bob do not need to be 
uniformly random, but sufficiently close to having high "entropy". To formalize this we 
introduce some standard definitions. 

We recall the statistical distancehetween V and V over sample space Q is ^^^i^filPi^vi^) — 
Pt'P'{u})\. We say a distribution V has min-entropy t the probability of every element is at 
most 2^*. A distribution T> is S -close to min-entropy t if there exists a distribution of min- 
entropy t which is within statistical distance 6 of V. Abusing notation, we will say that a 
function / : {0, l}" -^ {0, l}'^ has ((5-close to) min-entropy t if the distribution f{x), where 
X is uniform over {0, 1}", has ((5-close to) min-entropy t. 



Theorem 1. For any two functions f,g : {0, 1}" — !■ {0, l}'^ that are S-close to min-entropy 
t and every e < 1/2, 

Pr(.,,)J/(x) = (?(y)]< 2^*^/(1-) + 25. 

In particular, if the output of Ahce and Bob is exactly uniform then k = t and 6 = 0, 
so if they both output 1/e common bits they cannot hope to agree with probability better 
than 1/2. 

To prove the theorem, we will use the following two well known claims. Claim 2 follows 
from the fact that E(^x,y)Af i^)9iy)] i^ ^^ inner product of / and g. Claim 3 is a corollary of 
the hypercontractive inequality [Bon70, Bec75] as it is used in [KKL88]. The proofs of these 
claims require some additional notation. We first show how they imply the theorem. 

Claim 2. For every pair of functions f,g : {0, 1}'" — ?■ M, 

E(x,j/)J/(a;)^(y)] < ^jE(^^.y)Af{x)f{y)]E(^^^y)Ag{x)g{y)]. 

Claim 3. For every function h : {0,1}" -^ {0,1}, E[h{x)h{y)] < E[h{x)Y/^^^-'l 

Proof of Theorem 1. Assume first / and g have min-entropy t. For every z G {0, l}'^, let 
f, : {0, 1}" -^ {0, 1} be the function 

lo, otherwise. 

Define g^ similarly. Then E[/2(x)] and E[(72(x)] are upper bounded by 2^*. Therefore 

Vi[f{x)=g{y)]= J2 Pr[f{x) = zAgiy) = z] 
ze{o,i}'= 

= Yl nM^)9M] 

< Yl VnUx)fM]-n9z{x)gM] byClaim2 

zejo,!}*^ 



< Y VE[/,(x)]V(i-) . y^E[^,(x)]V(i-) by Claim 3 
zejo,!}'' 

< / J2 E[/,(x)]i/(i-^) . / J2 E[^,(x)]V(i-^) by Cauchy-Schwarz 
y ze{o,i}'' \j ze{o,i}''- 

Since / and g have min-entropy t it follows that Pz = E[fz{x)] < 2~* and similarly for g. We 
can now bound the expression in the first square root by 

Y p'J^'~'^= Yl p. X pf ^"^) < 2-*^/(i-^) Y p. = 2-*^/^'"'^ 

zG{0,l}'= zG{0,l}* ze{0,l}'' 



By an analogous calculation for the second expression, we obtain that Pr[/(x) = g{y)] < 

2~te/{l-e)_ 

In the case where / and g are 5 close min entropy t distributions we proceed as follows. 
Let 5' > 6. Then by possibly taking a larger value of n, there exist /' and g' of min entropy 
t such that such that Pr[/ ^ /'] < 5' and Pr[5f ^ g'] < 5'. Now: 

Pr[/(x) = giy)] < Pr[/'(x) = g'iy)] + Pr[/(x) ^ fix)] + Filgiy) ^ g'iy)] < 2^*^/(1^^) + 25'. 

Since (5' > (5 is arbitrary the proof follows. D 

We now prove the two claims. For this we make use of the Fourier expansion of Boolean 
functions: Every function / : {0, 1}" — )■ M can be uniquely written as 



fix) = X^ /s ■ Xs{x) 



SC[n] 

where the character functions Xs ^-re given by 

Xsix) = i-l)^^^s^^, 

The characters are orthonormal with respect to the inner product (/, (?) = E[f{x)g{x)]. 
It follows by a calculation that 

E(^^,y^Af{x)9{y)]=J2fs9sp'^'^ (1) 

SC[n] 



where p = y/1 — 2e. 

Therefore, to prove Claim 2 we observe that 

SC[n] 



< / y^ fsP^^^^ ■ / 2_^ 9sP'^^^^ by Cauchy-Schwarz 

SC[n] y SC[n] 



E(^,y), [f{x)f{y)] E(^,y)^ [g{x)g{y)]. 

To prove Claim 3, we make use of the hypercontractive inequality [BonTO, Bec75]. This 
inequality states that for every function / : {0, 1}" — )■ M, we have 

E[((T,/)(x))T/' < E[/(x)i+^Y/^'^'''^ (2) 

where Tpf : {0, 1}" — )■ M is defined via the Fourier expansion of / as the function 

(T,/)(x)= J2 fsP^'^Xsix) 

SCln] 



Comparing this with (1), we have that 

E.[((r,/)(x))2]=E(.,,)J/(x)/(y)] 



Where p = -y/l — 2e. Now, applying the hypercontractive inequahty (2) to a function h : 
{0,1}" -^ {0,1} we obtain 

which proves Claim 3. 

3 A better strategy 

We now show that when the agreement probability is sufficiently low, the trivial strategy 
can be outperformed, and in fact one can get strategies that approach the upper bound from 
Theorem 1 to within a constant factor. 

Theorem 4. Assume k > 10 + 2(1 — e)/e, and let n = n{k,e) be sufficiently large. There 
exists a function f : {0, 1}" — > {0, l}'^ such that for all z G {0, 1}^' it holds that 

yz e B^ Pr[/(x) = z] = 2-^ V^ G B^ Pr[/(x) = f{y) = z\f{x) = f{y)] = T\ 

Pr(.,.)J/(x) = f{v)\ > 0.003(eA;)-^/22-'=^/(i-) 

The protocol has the following form. Before starting, Alice and Bob agree on a subset C 
of {0, !}"■ of size 2''. On input x (respectively y), Alice (respectively Bob) finds and outputs 
the index of the closest point in C (with an explicit rule in case of ties). We will show that 
there exists a choice of C for which (1) each output is generated with the same probability 
and (2) the probability of agreement is high. 

In fact, we prove that on average, a random subspace of {0, 1}" of dimension k has both 
properties (1) and (2). In our analysis, we fix k and the noise e and let n go to infinity. 

Let C be an affine subspace of {0, 1}". Write C = a + L where L is a linear subspace. 
Let -< define a strict total order on {0, 1}" with the property that if the Hamming weight of 
X is smaller than the Hamming weight of y then x ~< y. We define the regions Re, c E C by: 

Re = {x: X + c ^ X + c' ioT all c j^ c E C} 

Note that if c is the unique closest point to x among all the points in C then x G Re- 

Let H : L -^ {0, 1}'' be any invertible linear map and let / : {0, 1}*^ — )■ {0, l}'^ be defined 
as f{x) = H{c), where c is the unique point such that x G Ra+c- 

Claim 5. For all z G {0, 1}'', 

Pr[/(a;) = z] = 2"'= and Pr[/(x) = f{y) = z\f{x) = f{y)] = 2~K 



Proof, li a + c, a + c' & C and x + c & Ra+c then x + c' E Ra+c' ■ So for every c E L we have 
f{x + c) = f{x) + H{c). Let z, z' G {0, l}'^ and let z' = z + H{c) where c E L. Then (x, ?/) 
and (x + c, y + c) have the same distribution and therefore 

Pr[/(x) = f{y) = z!\ = Pr[/(x + c) = f{y + c) = z + H{c)] = Pr[/(x) = f{y) = z], 

and similarly 

Pr[/(x) = z'] = Pr[/(x) = z + H{c)] = Pr[/(x + c) = z] = Pr[/(x) = z], 

as needed. D 

Let t be chosen so that 

^ / e-'"/'dz = 2-^-2 

Let C be a random affine space in {0, 1}" of dimension k. Let r = n/2 + ty^/2 and note 
that by the central limit theorem the hamming ball of radius r contains 2"~'^"^(1 — o(l)) 
points as n — 7- oo. We will say x G {0, 1}" is covered by c G C (denoted by x G Be) if x 
belongs to the ball of radius r centered at c. We say x is uniquely covered by c (denoted by 
X G Uc) if it is covered by c but not by any other c' G C. Observe that Uc ^ R^. 

Claim 6. Let C be a random affine subspace of {0, 1}" of dimension k. Then for n suffi- 
ciently large, 

EcPr(,,j,)J3c G C: x,y G f/e] > ^ ■ Pr[Z > ^^/(l - E)t] 

o 

where Z is a normal variable of mean and variance 1. 

By Claims 5 and 6, there must exist a set of points C for which (1) all the regions Re 



and of the same size and (2) Pr(a, j^) Jx, y G Re for some c G C] > | ■ Fi[Z > ^/e/{l — e)t]. 
To finish the proof of Theorem 4 we calculate a lower bound for the last expression. 

Claim 7. Let A; > 10 + 2(1 - 6)/e. Then 

I ■ Ft[Z > y/e/{l-e)t] > 0.003(£A;)-i/22-^'=/(i--) 
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where Z is a normal variable of mean and variance 1 . 

Proof. We will use the following estimates valid for every y > 0: 

y e-y'/' < / e-^'/'dz < -e~y'l\ 



y^ + 1 " L ' '" y 



'y 
Note that if A; > 10 then t > 3 and therefore 



1 /"oo 

J- / ^2, 



V27r A V27rt 



which imphes that t < \/2{k + 2) ln2 < y^k. Moreover 






which imphes that t > \/k\n2. So if fc > 10 + 2(1 — e)/e, then t > a/(1 — e)/e and therefore 

> i ■ ^ . (g-tV2^^/(l-^) 



27r 2v/£/(l-£)i 

> ^ . ^ 9-e(fc+2)/(l-£) 

~ v^ 2v/2£/(l-e)A; 

> 0.024 ■ (£A;)-^/22-efc/(i-e) 

as needed. D 

Proof of Claim 6. Let C be a random affine /c-dimensional space of {0, 1}". Such a space can 
be constructed by starting with a random point Cq ~ {0, 1}", and iteratively constructing 
the space Cj = cq + span(co + ci, . . . , cq + q), where q is chosen uniformly from {0, 1}" \ Ci-i. 
Finally let C = Ck- From the construction of C it follows that AC + c has the same 
distribution as C for every invertible linear transformation A and every vector c. Since for 
every pair of vectors a ^ a\h ^ h' there exsits an invertible A and a c such that Aa = b and 
Aa' = b' it follows that Prc[a, a' e C] = Pre [6, b' e C]. Then 

Ec7Pr(a.,j;),[3cGC: x,yeUc] 

c£C 



c£C 

> Ec ^ Pr(^,3/)^ [x, y e 5c] (^1 - ^ Pr(x,y), [x e 5c' or y e 5c' I X, y e 5c] j 
cec c'^c 

= ^ Ea^{o,i}^ Pr[x, y e 5„] (1 - X^ Ea'^{o,i}'^\{a} Pr[x G 5„' or y G 5„' | x, y G 5^] 

{0,1}*= a'T^a 

The last line uses the fact that the distribution over any pair of points c 7^ c' in a random 
affine space (of dimension at least 1) is the same as the uniform distribution over pairs 
a, a' G {0, 1}" conditioned on a' 7^ a. For the expression in the inner summation, we have 

Ea'~{o,i}" Pr(x,y)Ja; G 5^' or y G 5^' | x, y G 5^] 

< 2E^'^{o,i}" Pr(x,y)Ja; G 5„' | x,y G 5„] = 2Pr^[x G Bq] < 2~^-^ 



or 

fc-1 ^ 



and therefore 

Ea'^{0,l}n\{a} Pr(^,y)Jx, y G 5a/ I X, y G fia] < 2"''' _ 

from where the desired expression equals at least 

on 

Y, E„Pr(.,,)Jx,y G fij ■ (1 - (2' - 1)2~'-'^;^-^) > 2^ ■ E, Pr(.,,) Jx, y G B,] ■ (1/2) 

{0,1}'' 

= 2'=-i.Pr(,,,)Jx,yGfio]. 

To calculate the last expression, by the two-dimensional central limit theorem we have 

Pr(x,y), k, y e Bo]^ Prx,z[X >t,eX + Vl-O^Z > t] as n ^ oo 

where 6 = 1 — 2e and X, Z are independent normal variables with mean and variance 1. 
We now lower bound this expression: 

Frx,z[X >t,eX + Vl - e^Z >t]= Pr[X > t] Fr[9X + Vl-9^Z > t | X > t] 



> Pr[x > t] Piiet + Vi-e^z > t] 

= Ft[X > t] Ft[Z > y/e/{l-e)t]. 
Recalling that as n — )■ oo, Pr[X > t] — ?> 2"*"'"^, we obtain that as n becomes sufficiently large, 



Ec Pr{x,y), [x, yeUc for some c e C] > - ■ Ft[Z > Ve/(1 + 6)t]. D 

o 

4 Conclusion 

In this work we propose the following protocol for two parties that are given access to n 
noisy random bits with noise of rate e to agree on a common random string of length k: 

Preprocessing stage: 

1. Define a strict total order -< on {0, 1}" that is consistent with the partial order 
induced by Hamming weight. 

2. Choose a random fc-dimensional affine subspace C of {0, 1}". Identify the elements 
of C with strings in {0, l}'^. 

Decoding stage: On input x, output the unique c & C such that a; + c -< a; + c' for 

all c' eC,c' ^ c. 

Our analysis shows that on average over the choice of C, the outputs of Alice and Bob 
agree with probability D.{{ke)''^''^2~^'^'^^"'^'), which is best possible up to a factor of 0{vke) 
provided that k > 2/e + 0(1) and n = n{k, e) is sufficiently large. 

We remark that an explicit upper bound on n in terms of k and e can in principle be 
obtained by using a quantitative version of the central limit theorem in our arguments. 

We leave open the question of designing a deterministic and more efficient protocol for 
the problem considered here. It may also be interesting to investigate how much common 
randomness can be extracted from other noisy channels {x,y). 
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